$450,000 for digital orchestra conducting research

Announcement of NSF grant for digital conducting research

From the article:

Instead of conducting an orchestra from a concert hall, IU music informatics professor Chris Raphael will bring the orchestra to a computer screen.  Raphael received a three-year, $450,000 grant from the National Science Foundation for his proposal, “Real-Time Planning of a Conductable Orchestra,” said Lisa Herrmann, manager of communications for the School of Informatics. 

The culmination of the project will be a computer program that is able to understand the gestural language of musical conducting through video

Wow, $450,000?

I guess the NSF didn’t realize that this sort of thing has already been done and already works on generic computer technology. I have been doing this for a year now using a Mac and a Wii controller along with off-the shelf applications that convert the Wii’s signal to control the applications. All for less than $20,000. (See http://www.fauxharmonic.com )

And, of course, other researchers have been working in this area for years. Notably Teresa Marin Nakra, Jan Borchers and others. Hopefully the grantees will do a careful, comprehensive review of the state-of-the-art before forging an already-trod path. That would be a very helpful contribution to the field of digital orchestra research.

But, still, why should it cost so much? Maybe because the symbolic gestural “language” of conducting is not and has never been so clear cut that a digital signal processing system could “interpret” these gestures, and that, therefore, perhaps hundreds of thousands of dollars might begin to crack that nut?

The model of conducting that construes it as communicating instructions to the orchestra is only, at best, partial. A great deal of the gestural language of conducting is dance-like, in that it reflects the character of the music in visual form … for the audience as well as for the musicians. So conducting is no more “pretty well defined” than dance in terms of being able to be “understood” by a computer program.

And this large collection of gesture, body language, facial expression and conducting tradition is highly unlikely to be cracked by the IU researchers. More likely they will have to severely restrict what gestures their system will recognize as conducting. Conducting students will have to conform to the system, artificially cutting themselves off from a vast repertoire of subtle aesthetic expression in order to get the technology to play faster, slower, louder or softer.

If that’s how it turns out, it will be a much poorer tool for training actual conductors than, say, conducting a student ensemble (which $450k could support for the next 20 years at IU).

And what’s with that souped-up Boesendorfer piano that has nothing to do with orchestra (digital or otherwise), and costs well over $100,000?  Wouldn’t a Yamaha Disklavier do just as well? Or, even more a propos the subject: an installation of the Vienna Symphonic Library?

Well, it will be fascinating to see what results come of this research. Hopefully Raphael will avoid the dead-ends of earlier work in this area over least 30 years and come up with something that offers truly responsive musical expression, and not just a fancier way to twiddle with the volume and speed knobs on a recording.